1. Field of the Invention
The present invention relates to artificial neural networks. More particularly, the present invention relates to a multi-layer array of electrically-adaptable autocompensating amplifiers for use in a two-layer neural network.
2. The Prior Art
Several schemes for using a matrix of electronic devices for neural network applications have been proposed. To date, all such schemes involve using "weights" to control the amount of current injected into an electrical node "neuron". In most prior art structures, these weights were set by controlling the value of a resistor or the saturation current of a transistor. The limitation of any such scheme is that the value of any parameter of an electronic device in an integrated circuit is not well controlled. For example, the saturation currents of two MOS transistors of the same size can differ by a factor of two if these devices are operated in the sub-threshold regime. The mechanism that adjusts the weights must take these uncertainties into account.
U.S. Pat. No. 5,083,044 discloses and claims a synaptic element comprising an adaptive amplifier. The amplifier incorporates a floating gate element and may be adapted by exposing a portion of the floating gate of the adaptive amplifier to a source of ultraviolet light. This synaptic element may be used as a trainable synapse in which the weights may be adjusted to compensate for typical transistor nonuniformities and to otherwise manipulate the weights.
It is desirable to provide an adaptive mechanism which may be adapted by electrical means whereby the amplifier electrically adjusts itself to any uncertainty in device parameters, as part of the training process. Such a synaptic element is disclosed in parent application Ser. No. 07/525,764, filed May 18, 1990, now U.S. Pat. No. 5,059,920. This synaptic element has the advantage that it may be electrically adapted while the circuit is in its normal operating regime.
Single layer neural networks utilizing autocompensating amplifiers have been taught in the prior related applications listed above. Single layer networks are limited in that they cannot perform solutions to problems which are not linearly separable. This means that there is a large set of simple problems which cannot be solved by single layer networks because it is too difficult for them to map their input space into their output space. It has been proven mathematically that by adding a single layer to a neural network, the capability may be provided to map any function.
The present application extends the disclosure of the parent application to include a two-layer electrically-trainable neural network.